Visualization of Reputation Ratings

ABSTRACT

In one embodiment, a system and method is illustrated including receiving a feedback request identifying a particular user, retrieving a feedback entry in response to the feedback request, the feedback entry containing a first term, building a scoring model based, in part, upon a term frequency count denoting a frequency with which the first term appears in a searchable data structure, mapping the first term to a graphical illustration based upon a second term associated with the graphical illustration such that the graphical illustration may be used to represent the second term, and generating a feedback page containing the first term and the graphical illustration. The method may include assigning a value to the first term so as to identify the first term, assigning the first term to the searchable data structure, and extracting the first term from the searchable data structure based, in part, upon an extraction rule.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/167,289 filed on Oct. 22, 2018, which claims priority to U.S. patentapplication Ser. No. 14/611,087 filed on Jan. 30, 2015, which claimspriority to U.S. patent application Ser. No. 11/834,817 filed on Aug. 7,2007, which claims priority to U.S. Provisional Patent Application Ser.No. 60/912,389 filed on Apr. 17, 2007, and U.S. Provisional PatentApplication Ser. No. 60/912,077 filed on Apr. 16, 2007. The benefit ofpriority of each of these applications is claimed hereby. Further, thedisclosures of these applications are incorporated by reference hereinin their entirety.

TECHNICAL FIELD

The present application relates generally to the technical field ofsearch algorithms and, in one specific example, to the use of a searchalgorithm to generate feedback.

BACKGROUND

Customer feedback for online transactions allows potential purchasers ofgoods or services to evaluate a seller of a good or service prior toengaging in a transaction with the seller. In some cases, this feedbacktakes the form of statements regarding a particular seller, a good orservice, or a category of good or service being sold. These statementsmay range form being very general to being very specific in terms of theinformation that they convey.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments are illustrated by way of example and not limitation inthe figures of the accompanying drawings in which:

FIG. 1 is a diagram of a system illustrating the generation of feedbackdata, according to an example embodiment.

FIG. 2 is a diagram of a system illustrating the generation of afeedback page, according to an example embodiment.

FIG. 3 is a Graphical User Interface (GUI) illustrating a feedback page,according to an example embodiment.

FIG. 4 is a GUI showing a feedback page with actual feedback for aparticular user having the user ID “Happy Shopper”, according to anexample embodiment.

FIG. 5 is a GUI illustrating a feedback page showing specificinformation with regard to a particular phrase, according to an exampleembodiment.

FIG. 6 is a GUI showing a feedback page illustrating negative feedback,according to an example embodiment.

FIG. 7 is a block diagram of some of the functional blocks that make upan application server implementing the method illustrated herein,according to an example embodiment.

FIG. 8 is a dual stream flowchart illustrating a method used to generatefeedback and then retrieve this feedback for the purpose of generating afeedback page, according to an example embodiment.

FIG. 9 is a flowchart illustrating a method used to execute an operationthat parses and stores feedback data into a feedback data store,according to an example embodiment.

FIG. 10 is a flowchart illustrating a method used to execute anoperation to retrieve feedback entries from a feedback data store,according to an example embodiment.

FIG. 11 is a flowchart illustrating a method used to execute anoperation that filters certain noise words, according to an exampleembodiment.

FIG. 12 is a flowchart illustrating a method used to execute anoperation that assigns words to an array, or some other suitable datastructure, according to an example embodiment

FIG. 13 is a flowchart illustrating a method used to execute anoperation that assigns each word to some type of searchable datastructure, according to an example embodiment.

FIG. 14 is a diagram of a plurality of tries containing feedback data inthe form of clean comment text, according to an example embodiment.

FIG. 15 is a flowchart illustrating a method used to execute anoperation to extract certain phrases from a searchable data structureand to pass these phrases through a frequency engine, according to anexample embodiment.

FIG. 16 is a flowchart illustrating a method used to execute anoperation that builds a scoring model using a frequency count, accordingto an example embodiment.

FIG. 17 is a diagram illustrating the conversion of the previouslyillustrated tries into a data structure (e.g., a hash table), accordingto an example embodiment.

FIG. 18 is a flowchart illustrating a method used to execute anoperation that compares the frequency count for each of the phrasesagainst certain graphic standards relating to particular graphicalillustrations (e.g., emoticons), according to an example embodiment.

FIG. 19 is a flowchart illustrating a method used to execute anoperation that generates the feedback page, according to an exampleembodiment.

FIG. 20 is a Relational Data Schema (RDS) illustrating some of theexample database tables that may be used in one or more embodiments ofthe system and method illustrated herein, according to an exampleembodiment.

FIG. 21 shows a diagrammatic representation of a machine in the form ofa computer system, according to an example embodiment.

DETAILED DESCRIPTION

Example methods and systems to facilitate feedback ratings aredescribed. In the following description, for purposes of explanation,numerous specific details are set forth in order to provide a thoroughunderstanding of example embodiments. It will be evident, however, toone skilled in the art that the present invention may be practicedwithout these specific details.

In some embodiments, a system and method for providing feedback to apotential purchaser of a good or service is illustrated. This system andmethod may provide specific and unique feedback (e.g., feedbackratings), which, in some cases, is devoid of redundant, cumulative, orother types of information that do not contribute to educating apotential purchaser of a good or service about either the good orservice or about the party selling the good or service. Further, thissystem and method may provide a potential purchaser with one or morevisual clues as to the quality of the purchasing experience that actualpurchasers have had with the particular party selling the good orservice, and/or the good or service actually sold. In one embodiment,these visual clues may take the form of a graphical illustration (e.g.,an emoticon), and/or various textual highlights displayed on a feedbackpage in the form of, for example, a Hyper Text Markup Language (HTML)based webpage.

Providing generalized feedback ratings for marketplace participantsoften lacks the specificity to be informative for a user. Examplemarketplace feedback scores may give a quantitative measure of usertrustworthiness, but they may at the same time lack the requisite detailto be informative. This may be true even where the feedback may becategorized to include a plurality of positive, negative and neutralcomments. More to the point, pure feedback scores, even if categorizedby positives, negatives, and neutrals, may not differentiate betweenusers.

While feedback may give a sense of how good or bad users are (e.g., interms of the quality of service or goods they provide), it does notdescribe why the users may be good or bad. In some cases, it may bedifficult to tell what qualities differentiate one buyer from anotherand one seller from another. For example, it may be important to knowwhether a particular market participant is good, for example, atcommunication, packaging, pricing or service. Further, in some cases,users (e.g., those who have actually purchased a good or service from aseller) may leave neutral or positive feedback to avoid confrontation ormay put in information that relates to the quality of their businessconduct. In some cases, without a proper tool to differentiate thefeedback, potential customers may have to go through pages of texttrying to glean particulars.

Some example embodiments may include providing potential purchasers withan ability to automatically extract representative textual phrases ortags from a marketplace feedback text. Once the feedback tags areextracted, graphical representations in the form of, for example,emoticons, may be attached to the feedback tags. Multiple feedback tagsmay have the same emoticons attached to them revealing the sentiments ofusers who have actually purchased a good or service from the seller.

In some embodiments, technology may be implemented that includes thevisualization of reputation ratings by parsing the feedback andanalyzing its text for specific pattern frequencies. Text size may beused, for example, to show popularity of feedback by displaying morefrequently-used phrases in a bigger font. In other example embodiments,other visual differentiation techniques may be used to highlight ordistinguish more frequently-used (or otherwise identified) phrasesincluded in the feedback data. Uniqueness of the particular feedback isalso considered. An emoticon icon (e.g., emoticon) may be generatedbased on the feedback and may be displayed.

Further, in some embodiments, a tool may provide users with a legend ofemoticons used to visualize the feedback. The emoticons may berepresentative of emoticons specific to the feedback provided by aparticular community and may be displayed automatically. Moreover, usersmay be given an option of using cached information, and may selectwhether the feedback information should come from the item category orfrom user feedback. Additionally, in some embodiments, users may beprovided with hints to further explain the meaning of the icons andterms.

Example Technology

In some embodiments, the example technology may include allowingsearching by a user (e.g., a seller of goods or services), an identifier(e.g., a screen name, handle, or numeric identifier), or allowing a userto pick a user name from cached user names. The potential purchaser mayselect and search for visualized feedback that may, for example, bepresented in positive, negative, or neutral categories. Additionally,the potential purchaser may use emoticon legends (e.g., emoticonlegends) that correspond to the feedback terms. Further, the morefrequently-occurring feedback terms may be displayed in correspondinglylarger font. Example embodiments may include using displayed terms orphrases which are clickable to expose further details about the feedbackassociated with these terms or phrases. Moreover, the technology mayinclude demonstrating the frequency of use of the term or phrases bydisplaying the percentage of occurrence in the feedback.

Some example embodiments may include extracting representative textualphrases from a user's feedback text as “tags”. These tags may beextracted to differentiate feedback for one user from that of another.For example, if all users have the text “AAAAA++++” in their feedback,this information does not serve to distinguish one user from another. Insome embodiments, the example technology may extract otherdistinguishing phrases that summarize a user's (e.g., a seller's)feedback text. This information may be extracted at a global level, acategory level, a domain level (e.g., static or dynamic), or at anyother suitable level. Also, this information may be extracted for allthe transactions that have occurred for all users in a given category todescribe the most representative tags for that category feedback. Forexample, “very cute” may be a typical phrase in a positive feedback textin a bag category, “does not fit” may be a common phrase in an apparelcategory, and “wrong size” may be a common phrase in a jewelry-ringcategory.

Once the text is extracted, the information may be presented at a userlevel, across all categories, at a category level, domain level, or atsome other suitable level. Example representations of the feedback maybe shown with the text differentiated by its size (e.g., highlightedwith larger text for more frequent phrases and smaller text for lessfrequent ones).

Further, in some embodiments, an example technology may represent textemoticons or sentiments that may have emoticons attached to them. Thismay be done by keeping a dictionary of phrases and mapping the phrasesto a static set of emoticons. For example, “speedy shipping” and “fastdelivery” may be associated with a common emoticon. Linguistic analysisand natural language processing may help to identify such similarphrases and map them to the same emoticon. Techniques of sentimentmining from small text may be used to attach sentiments to the tagphrases.

Example System

FIG. 1 is a diagram of an example system 100 illustrating the generationof feedback data. Shown is a reviewer 101 who, utilizing a GUI 112,generates positive feedback. GUI 112 may reside on any one of a numberof devices 102 including, for example, a cell phone 103, a computersystem 104, a television 105, and/or a Personal Digital Assistant (PDA)106. Utilizing any one of these devices 102, the reviewer 101 maygenerate feedback data 126. This feedback data may be related to userswho buy and sell good and services, feedback related to a category ofsellers, buyer, good, services, or feedback relating to some othersuitable transaction. Feedback data 126 may, in some cases, be positivefeedback such as, for example, that a purchase the reviewer 101 made was“Best Purchase Ever”. Further illustrated is a reviewer 124 who,utilizing the GUI 112, may generate neutral feedback. This neutralfeedback may be in the form of, for example, feedback data 127, such asa statement that “Service Was About Average”. Further, a reviewer 125may generate negative feedback utilizing the GUI 112. This negativefeedback may be, for example, feedback data 128 which states, forexample, “Not A Good Experience” with regard to a particulartransaction. The feedback data 126, 127, and 128 may be transmittedacross a network 150 to, for example, a web server 130. Web server 130may be operatively connected to, for example, an application server 131.Operatively connected to application server 131 may be, for example, afeedback data store 132. In certain embodiments, a plurality of webservers 130 may be used to receive feedback data, such as feedback data126, 127 and 128. Further, a plurality of application servers 131 may beoperatively connected to one or more of these web servers 130.Additionally, a database server (not pictured) may be operativelyconnected to the application server 131 and may be used to retrieve datafrom and store data to the feedback data store 132.

FIG. 2 is a diagram of an example system 200 illustrating the generationof a feedback page. Illustrated is a reviewer 201 who, utilizing a GUI112, generates a feedback request and receives a response to thisfeedback request in the form of a feedback page. For example, asillustrated, a feedback request 202 may be generated by the reviewer 201utilizing the GUI 112. Feedback request 202 may, for example, containinformation identifying a particular user (e.g., a seller who hastransacted business on a particular website managed by a web server andassociated application server). Feedback request 202 may be transmittedacross the network 150 to the previously shown and illustrated webserver 130, and may ultimately be processed by an application server131. In response to the feedback request 202, a feedback page 203 may begenerated. Feedback page 203 may be, for example, a web page writtenusing HTML combined with certain technologies such as AsynchronousJavaScript and eXtensible Markup Language (AJAX), Dynamic HTML (DHTML),or even Active Server Pages (ASP). Feedback page 203 may then betransmitted by the web server 130 back across the network 150 forviewing on the GUI 112 by the reviewer 201. The process for generatingfeedback page 203 may be more fully illustrated below.

Example Screen Shots of Interfaces

In some embodiments, a feedback page 203 is displayed to a potentialpurchaser. Feedback page 203 may contain feedback regarding a particularseller of a good or service, or regarding a particular category of goodsor services for sale. In one embodiment, when a potential purchaserclicks on a tag underlying the text displayed on the feedback page 203,an asynchronous request is sent to the server to obtain details for thattag. This asynchronous request may be generated using technologyincluding, for example, AJAX, or DHTML. Once the asynchronous request isreceived, then feedback information related to that tag is extractedfrom a pool of feedback for that user. Next, in some cases, a percentageis computed to determine how many comments out of the total pool ofcomments (e.g., for that particular seller) actually relate to thatspecific tag. Then, in some embodiments, a feedback servlet againconstructs the HTML to be displayed to the client (e.g., displayed aspart of the feedback page 203). In some example embodiments, synchronoustransmissions of web page queries are utilized in lieu of or incombination with the asynchronous queries. Further, technologies such asASP may be utilized in lieu of servlets.

In some cases, a potential purchaser may generate a feedback requestrelating to comments regarding a seller, comments regarding anotherpotential purchaser, a feedback score, a category of items for sale, aparticular item, or some other type of suitable information pertinent tothe sale of a good or service. This information may be supplied bypurchasers or by others having access to this type of information.

FIG. 3 is a GUI displaying an example feedback page 203 relating tocategory feedback. Shown is a legend outlining a number of graphicalillustrations (e.g., emoticons). An emoticon 301 denotes that someonehas been pleased with a particular experience that they have had with auser (e.g., a seller). An emoticon 302 is shown illustrating that aparticular user has shipped something to a customer quickly. An emoticon303 illustrates that a particular customer has had an unhappy ordisappointing experience with a user. An emoticon 304 denotes that aparticular user may have a bad attitude. Finally, an emoticon 305denotes that a particular user may be average in terms of the experiencethat they have generated for a particular customer. Also shown is ascreen object or widget in the form of a textbox 306 that allows areviewer, such as reviewer 201, to insert a particular user identifieror user identifying information into this textbox 306. This useridentifier may be some type of unique numeric value or may be some typeof unique handle that may be used to identify a particular seller. Alsoillustrated is a menu 307 showing various names of sellers of variouscategories of goods. Shown here is a category of goods (e.g., a cache ofusers) in the form of “Women's Clothing-Outerwear” and various userhandles associated with this category. In some embodiments, these userhandles are related to individuals who are selling women's outerwear.

FIG. 4 is a GUI displaying an example feedback page 203 showing actualfeedback for a particular user having the user ID “Happy_Shopper”. Shownare a positive feedback field 401, a negative feedback field 405, and aneutral feedback field 408. With regard to the positive feedback field401, a number of phrases and associated emoticons are illustrated. Forexample, the phrase “great communication” (referenced here as 402) isshown, which is in turn associated with an emoticon 403 denoting thatsomeone is pleased (e.g., the previously-referenced emoticon 301). Thephrase 402, in this particular example, appears in bold text to denotethat this person has received a number of praises for facilitating greatcommunication in a transaction beyond some particular threshold value.This notion of threshold value, as it relates to bolding or otherwisehighlighting a particular phrase, will be more fully discussed below.Also shown is a phrase 404 that is not bolded or otherwise highlighted,wherein the user “Happy_Shopper” has received praise for selling“absolutely beautiful” items. The positive feedback field 401 has 1,402entries associated with it. Also shown is the negative feedback field405, illustrating negative feedback that the user “Happy_Shopper” hasreceived. There are 35 negative feedback entries for this user, asindicated by the number displayed in field 406. For example, the user“Happy_Shopper” has been referred to by the phrase “no response,” andhas also been referenced by an emoticon 407, indicating that certainparties have been unhappy or disappointed with the user “Happy_Shopper”(e.g., emoticon 303). The phrase “no response” is bolded or otherwisehighlighted to denote that a number of negative feedback responses orphrases illustrating negative feedback have been used to reference thisparticular user. Further, the neutral feedback field 408 is also shownwherein, for example, the user “Happy_Shopper” has provided the wrongsize to one or more customers. The feedback fields 401, 405 and 408, andthe data that is entered therein, may be supplied with data by certainreviewers, such as reviewer 101, 124 and 125. This process forgenerating a feedback page may be more fully illustrated below.

FIG. 5 is a GUI displaying a feedback page 203 showing specificinformation with regard to a particular phrase. Illustrated is a phrase501 that has been selected using a mouse pointer 502. Once the phrase501 is selected, a screen widget or object containing the specificfeedback that is used to generate phrase 501 is shown. Along with thisspecific feedback, the name or other identifying information of theparty who generated the feedback is also provided. Here, for example,the phrase 501 that states that goods have been “received in goodcondition” is generated by, for example, the various pieces of feedbackthat are displayed in a screen object or widget 503. Also, the feedbackprovided in this screen object or widget 503 pertains to feedbackrelating to when items sold were “received in good condition”, whichhere states that such feedback was only provided 0.1% of the time. Putanother way, this good feedback is actually bad feedback where viewedinto terms of the universe of data against which it is measured. Screenobject or widget 503 may be, for example, a text display box.

FIG. 6 is a GUI displaying an example feedback page 203 illustratingnegative feedback. Shown is a phrase 601 that states “no response.” Anemoticon 602 is associated with phrase 601, and denotes that one isunhappy or disappointed with a particular service they received from,for example, the user “Happy_Shopper”. Using a mouse pointer 502, thephrase 601 may be selected such that a screen object or widget 603 isgenerated containing the various feedback data used to generate thephrase 601 and user IDs associated with the feedback data. In somecases, screen object or widget 603 may show a certain percentage valuerelating to the percent of total negative feedback that describes theuser “Happy_Shopper” as having provided no response to customer queries.

Example Method

FIG. 7 is a block diagram of some of the functional blocks that make upthe application server 131. These blocks may be implemented in hardware,firmware, or software. Illustrated is a first receiver 701 to receive afeedback request identifying a particular user, a first retriever 702 toretrieve a feedback entry in response to the feedback request, thefeedback entry containing a first term, a score builder 703 to build ascoring model based, in part, upon a term frequency count denoting afrequency with which the first term appears in a searchable datastructure, a mapping engine 704 to map the first term to a graphicalillustration based upon a second term associated with the graphicalillustration such that the graphical illustration may be used torepresent the second term, and a page generator 705 to generate afeedback page containing the first term and the graphical illustration.Also shown is a first assigner 706 to assign a value to the first termso as to uniquely identify the first term, a second assigner 707 toassign the first term to the searchable data structure, and a termextractor 708 to extract the first term from the searchable datastructure based, in part, upon an extraction rule. Moreover, thefeedback entry may include at least one of positive, neutral, ornegative feedback. A highlighter engine 709 is also shown thathighlights the first term based upon the scoring model being the productof the term frequency count and an inverse document frequency value.This highlighter engine 709 may use a highlight including at least oneof a font size highlight, a color highlight, an underline highlights, ora bold highlight. The searchable data structure may include at least oneof a trie, hash table, red-black tree, or Binary Search Tree (BST). Thegraphical illustration may be emoticons. A filter 710 is shown thatfilters noise words from the feedback entry. Also shown is a secondreceiver 711 to receive an asynchronous feedback request generated, inpart, through selecting a phrase appearing on the feedback page, asecond retriever 712 to retrieve feedback information from a pool offeedback relating to the particular user, and a transmitter 713 totransmit the feedback information to be displayed on the feedback page.In some cases, the first term is part of at least one of a categorylevel context, a global level context, a static domain level context, ora dynamic domain context.

FIG. 8 is a dual stream flowchart illustrating an example method 800used to generate feedback and then retrieve this feedback for thepurpose of generating a feedback page. Illustrated is a first streamcontaining various operations that may reside as a part of, for example,the application server 131. These operations include operations 802,803, and 805-814. Also shown is a second stream containing variousoperations that may reside as a part of the one or more devices 102.These include operations 801, 804 and 815.

With regard to the first stream, an operation 801 may be executed thatgenerates feedback in the form of feedback data 126 that is thentransmitted to, or otherwise received through, the execution of anoperation 802. Once operation 802 is executed, an operation 803 may beexecuted that parses and stores feedback data 126 into feedback datastore 132. This process of generating feedback data and subsequentlyparsing and storing it into a data store may serve to, for example, seedor otherwise populate a data store with user feedback. In some cases,this user feedback may be subsequently utilized for the generation of afeedback page, its associated graphical illustrations (e.g., emoticons),and phrases describing a particular user in terms of feedback regardingthat user. Once feedback data store 132 is populated, a reviewer such asreviewer 201 may execute an operation 804 to generate a feedbackrequest, such as feedback request 202. This feedback may be generatedwith respect to a particular user, category of good, services, at astatic or dynamic domain context, or even for all transactions (e.g.,globally). Further, these types of feedback may be combined. Next,through the execution of an operation 805, feedback request 202 may bereceived and processed. Once feedback request 202 has been received, anoperation 806 may be executed that may retrieve feedback entries from,for example, the feedback data store 132. This operation 806 may usevarious Application Programming Interface (API) calls, or even callsgenerated using a Structured Query Language (SQL) to retrieve feedbackentries. Then, an operation 807 may be executed that generates a list ofpositive, neutral, or negative feedback entries. Put another way, insome embodiments, a list containing only positive feedback may begenerated, a list containing only neutral feedback may be generated, anda list containing only negative feedback may be generated. In lieu of alist, some other suitable data structure (e.g., a Binary Search Tree(BST), a stack, a queue, a double linked list) may be utilized.

Once these lists are generated, an operation 808 is executed thatfilters certain noise words, wherein these noise words may be containedin some type of predefined dictionary (e.g., a stop word dictionary).This dictionary may be based upon certain words that may be deemed to beuninformative or otherwise unhelpful in terms of facilitating areviewer's, such as reviewer 201's, understanding of feedback regardinga particular user. The execution of operation 808 may be optional insome embodiments. Once operation 808 is executed, or if operation 808 isoptionally not executed, the method 800 continues to an operation 809that assigns any remaining words to an array, or some other suitabledata structure, wherein a unique integer value is associated with eachone of the words. Then, an operation 810 may be executed that assignseach one of these words to some type of searchable data structure, suchas, for example, a trie, a BST, a heap, or a list. In some embodiments,a data structure may be generated for each feedback type (e.g.,positive, neutral, and/or negative feedback), resulting in a pluralityof data structures. An operation 811 may then be executed that extractscertain phrases from the searchable data structure (or plurality ofsearchable data structures) and passes these phrases through a frequencyengine. In some embodiments, the frequency engine counts the number oftimes these phrases appear in all of the searchable data structures, orin some cases, only one or more of the searchable data structures.

An operation 812 is then executed that builds a scoring model using thefrequency count. In some cases, this scoring model may, for example, bea hash table that contains a phrase and its frequency count based uponthe aggregation of the frequency values for the words associated withthe phrase. In some cases, this hash table may be implemented usingbucket hashing or cluster hashing as may be suitable. In other cases,some other suitable data structure may be used such as, for example, aBST, heap, linked list, or doubly linked list. Next, an operation 813 isexecuted that maps or compares the frequency count for each of thephrases to certain graphic standards relating to particular graphicalillustrations (e.g., emoticons). In effect, this comparison takes thephrase contained in a hash table entry and compares it to phrasesassociated with a particular emoticon. In some cases, a dictionary ofsentiments, or emoticon mining system (collectively referenced herein asan emotion database) is implemented such that the phrases are comparedto possible synonyms and where a match is found the correspondingemoticon is used. If, for example, the phrase “Best Purchase Ever” isconsidered to be synonymous with the phrase “Exactly Described”, thenthe emoticon 301 denoting that one is “pleased” with a transaction mightbe appropriate. Once the phrases are mapped to an emoticon, an operation814 is executed that generates a feedback page, such as feedback page203. The generation of feedback page 203 may be more fully illustratedbelow, but includes, for example, the generation of an HTML-based pagethat contains, for example, the phrases and their respective emoticons.In some embodiments, the phrases themselves may be highlighted in somemanner (e.g., font size, bolded font, italicized font, underlined font,color font, or some other suitable method of highlighting) where aparticular phrase may need to stand out relative to other phrasescontained in a particular field (e.g., a positive feedback field 401,negative feedback field 405, neutral feedback field 408). Someembodiments may include using a method of gradation based upon import tohighlight certain phrases such that the more unique the phrase isrelative to some universe of phrases, the great degree of highlightingit may receive. The method of gradation based upon import may be morefully discussed below is the section relating to the generation of theInverse Document Frequency (e.g., referenced as idf) value.

Feedback page 203 may then be received through the execution of anoperation 815 that may receive feedback page 203 and display it. In someembodiments, the execution of operation 815 may be carried out throughsome type of application capable of interpreting an HTML-based page,such as, for example, a web browser or other suitable application thatmay interpret, for example, HTML or XML. Further, through the executionof operation 815, details relating to a particular piece of feedback maybe displayed (see e.g., phrase 501 selected with mouse pointer 502 so asto display the specifics of the selected phrase in screen object orwidget 503).

FIG. 9 is a flowchart illustrating an example method used to executeoperation 803. Illustrated is an operation 901 that associates feedbackdata with a particular user ID, wherein this feedback data may be, forexample, feedback data 126, 127, or 128. Next, an operation 902 isexecuted that parses the feedback database based upon some type of datacategory. This data category may be, for example, whether the feedbackpertained to a particular seller or buyer, a particular category ofitem, a particular cost of an item or some other suitable category forwhich feedback may be provided. Next, an operation 903 is executed thatstores feedback data according to a data category. For example, feedbackdata may be stored according to whether it pertained to a particularuser, whether it was negative, positive, or neutral, or whether itpertained to a particular item, a cost of that item, or some othersuitable category. Through the execution of operation 803, as previouslyillustrated, a feedback data store 132 may, in effect, be populated withdata that may be used to generate a feedback page 203 at some laterpoint.

FIG. 10 is a flowchart illustrating an example method used to executeoperation 806. Shown is an operation 1001 that generates a databasequery using a user ID and a data category. In some cases, this databasequery may be, for example, an SQL based query, API based query, while inother embodiments, it may be some other suitable query language (e.g.,Multidimensional Expression (MDX) language) based query. Next, anoperation 1002 may be executed that retrieves feedback data using thedatabase query. This selection process, again, may be implementedutilizing some type of suitable query language and various methods oroperations associated with this query language (e.g., select, from,where). Then, an operation 1003 is executed that segregates feedbackdata by positive, neutral or negative data categories.

FIG. 11 is a flowchart illustrating an example method used to executeoperation 808. Illustrated is an operation 1101 that receives certainretrieved feedback data from a list of positive, neutral, or negativefeedback, or in some cases, from multiple lists of positive, neutral, ornegative feedback. The feedback data is, for the most part, text-baseddata. Then an operation 1102 is executed that cleans the feedback databy removing non-character elements. Non-character elements may be, forexample, certain graphical references or other types of non-text-baseddata. An operation 1103 is then executed that may optionally removenoise words from the feedback data by comparing the text containedwithin the feedback data to a dictionary of noise words. The dictionaryof noise words may be contained in, for example, a noise word database1104, and as previously discussed, may be generated specifically for thepurpose of excluding certain types of words. These types of words mayinclude, for example, words that may be profane in nature or mayotherwise not assist a reviewer 201 in understanding a feedback page 203and the data (e.g., text) contained therein. Next, an operation 1105 isexecuted that generates cleaned comment text.

FIG. 12 is a flowchart illustrating an example method used to executeoperation 809. Shown is an operation 1201 that receives clean commenttext. An operation 1202 is then executed that generates an array ofintegers, where each of these integers uniquely corresponds to a word inthe clean comment text. In some embodiments, a single array of integersis generated for all neutral, negative, or positive feedback. In otherembodiments, an array of unique integers is generated only for thepositive text, only for the neutral text, and only for the negativetext. Further, another array of unique integers is generated only forthe neutral text. In some embodiments, an array of unique integers isgenerated only for the negative text. Further, as illustrated elsewhere,some other suitable data structure containing this series of uniqueintegers may be generated such that, for example, a list, a stack, aqueue, or some other suitable data structure may be used.

FIG. 13 is a flowchart illustrating an example method used to executeoperation 810. Shown is an operation 1301 that receives clean commenttext and the previously shown array of integers. An operation 1302 isexecuted that extracts the clean comment text from each of the listswith which they are associated. Then, an operation 1303 is executed thatinserts the clean comment text into one of a positive, neutral, ornegative data structure, wherein this data structure may be, forexample, a trie, a BST, a red-black tree, a heap, a hash table, or someother suitable data structure that may facilitate the search andretrieval of this clean comment text. In some cases a clean comment textis inserted using a suffix ordering and the integers from the array ofintegers. In some embodiments, a prefix ordering may be used in lieu ofa suffix ordering. As will be more fully illustrated below, in oneembodiment, a trie may be implemented wherein each node of the trie isuniquely identified with one of the unique integer values, and the linksbetween each one of the nodes of the trie contain a particular word thatis a part of a phrase contained in the clean comment text.

FIG. 14 is a diagram of a plurality of tries 1400 containing feedbackdata in the form of clean comment text. Shown are a first sub-trie 1429containing positive feedback, a second sub-trie 1430 containing neutralfeedback, and a third sub-trie 1431 containing negative feedback. Thesevarious sub-tries are, in some cases, part of larger tries showingpositive, neutral, and negative feedback. With regard to the firstsub-trie 1429, positive feedback in the form of the phrase “BestPurchase Ever” is illustrated. Using a suffix ordering, this phrase maybe broken down such that a node 1401 (e.g., a root node) is created.This root, or parent, node has a number of child nodes, such as nodes1402, 1432 and 1403. These child nodes are linked to their parentthrough various edges, wherein the edge connecting nodes 1401 and 1402contain the word “Best”, the edge connecting nodes 1401 and 1432 containthe word “Purchase”, and the edge connecting nodes 1401 and 1403 containthe word “Ever”. Node 1402 has its own child node 1404. Connecting thenodes 1402 and 1404 is an edge containing the word “Purchase”. Node 1404also has a child node, 1406, where this child node 1406 is, in effect, aleaf node. Connecting the nodes 1404 and 1406 is an edge containing theword “Ever”. As to the node 1432, it too has a child node 1405 which is,in effect, a leaf node. Connecting nodes 1432 and 1405 is an edgecontaining the word “Ever”. Next a node 1403, which acts as a leaf node,is connected to the root node 1401 by an edge containing the word“Ever”. In one embodiment, the path from node 1401 to node 1406 isreflective of the previously-stated positive feedback (e.g., “BestPurchase Ever”). Subsequent paths or other paths through this firstsub-trie 1429 create other phrases such as, for example, “Purchase Ever”(e.g., the path between node 1401 and 1405), or even just a single wordbeing used to form a path (see e.g., the path between node 1401 and1403).

The second sub-trie 1430 provides an illustration of a suffix orderingrelating to neutral feedback in the form of the phrase “Service WasAbout Average”. Shown is a root node 1407. Connected to root node 1407are a number of child nodes including, for example, nodes 1408, 1409,1410, and 1411. Some of these child nodes themselves have children, suchthat, for example, node 1408 has a child node 1412, node 1409 has achild node 1413, and node 1410 has a child node 1414. Again, some ofthese child nodes also have children, such that, for example, node 1412has a child node 1415, and node 1413 has a child node 1416. Alsoillustrated is a leaf node 1417 that is a child of node 1415. Againtraversing from the root node to the leaf nodes, there are a number ofphrases that may be generated, such that, for example, traversing fromthe root node 1407 to the leaf node 1417, the phrase “Services Was AboutAverage” may be generated. Likewise, traversing from the root node 1407to the node 1416, the phrase “was about average” may generated.Similarly, traversing the path from root node 1407 to the node 1414, thephrase “About Average” may be generated, and traversing from the rootnode 1407 to the node 1411, the phrase, or in this case word, “Average”may be generated.

Further illustrated is a third sub-trie 1431 that relates to negativefeedback that a particular user has received, for example, in the formof the phrase “Not A Good Experience”. Shown as a part of trie 1431 is aroot node 1418 having a number of child nodes 1419, 1420, 1421, and1422. These child nodes themselves may have one or more children. Forexample, node 1423 is a child of the node 1419, node 1424 is a child ofthe node 1420, and node 1425 is a child of the node 1421. Other childrenillustrated herein include, for example, node 1426 as a child of node1423, and node 1427 as a child of node 1424. Additionally, a leaf node1428 is illustrated as a child of the node 1426. Trie 1431 may betraversed via following any one of a number of paths from the root nodeto various child or leaf nodes. The phrase “Not A Good Experience” maybe generated through traversing the path from the root node 1418 to thechild node 1428. Another path may be traversed from the root node 1418to the child or leaf node 1427 wherein the phrase “A Good Experience”may be generated. Another phrase, “Good Experience,” may be generated bytraversing the path from the root node 1418 to the leaf node 1425, andyet another phrase, or in this case word, “Experience” may be generatedby traversing the path between the node 1418 and the leaf node 1422. Asillustrated elsewhere, some other suitable type of data structure may beused in lieu of tries to organize and traverse strings and substringsassociated with positive, neutral, or negative feedback.

FIG. 15 is a flowchart illustrating an example method used to executeoperation 811. Shown is an operation 1501 that receives a data structurecontaining clean comment text. Once the data structure(s) have beenreceived, an operation 1502 is executed that retrieves certain phraseextraction rules from, for example, a phrase extraction rules database1503. Then, an operation 1504 may be executed that extracts phrases fromthe data structures (e.g., the tries) based upon the phrase extractionrules. Next, an operation 1505 is executed that passes the extractedphrases through a term frequency counter to generate a frequency countfor each of the terms. In some cases, the phrase extraction rules mayset certain rules regarding what constitutes a phrase, such that aphrase may only contain one, two, three, or some other predefined seriesof words. Further, the words that make up a phrase may only beextracted, in some cases, when they have occurred a certain number oftimes. This certain number of times may be one, two, three, or someother predefined number of times. The generation of the extraction rulesmay be done by, for example, a system administrator or some othersuitable individual, and may be stored as previously illustrated intothe phrase extraction rules database 1503.

FIG. 16 is a flowchart illustrating an example method used to executeoperation 812. Shown is an operation 1601 that receives extractedphrases and a frequency count for each phrase. Next, an operation 1602is executed that generates a scoring model for each phrase using, forexample, a frequency count. Once the scoring model is generated, anoperation 1603 is executed that stores the phrases, the respectivescoring models, and other data into a data structure such as, forexample, a hash table, an array, a list, or some other suitable datastructure. In certain cases, the scoring model itself may be the datastructure containing the extracted phrase as frequency count and otherdata. In other cases, a scoring model may just be, for example, data inthe form of a phrase and its associated frequency count. In someembodiments, the frequency count may be generated based upon anaggregation of accounting of each word that makes up the phrase suchthat, for example, if there are three words that make up a phrase andeach word has a frequency count of 60, then the frequency count for thephrase would be 180.

In some embodiments, another scoring model (e.g., a combined scoringmodel value) may be utilized in lieu of or in conjunction with thefrequency count. This combined scoring model value may be generatedbased upon, for example, the product of a term's frequency (e.g., itsterm frequency (tf), or frequency count), and the number of documentsthat contain that term for a particular user (e.g., a buyer or seller)as compared to a given universe of documents (e.g., the universe of alluser feedback) (e.g., idf value). In one embodiment, once the hash tableis built for all phrases extracted, the combined scoring model value isthen computed for each phrase using a tf value for global data, and anidf value for global data. In one embodiment, this tf*idf score may bestated as:

Combined scoring model value=tf*idf;

where tf=((number of times phrase “X” occurs for seller A)/(max termfrequency for seller A)); andidf=log 2((number of documents[e.g., feedback comments] in thedataset)/(number of times phrase X occurred in the entire dataset)).In some embodiments, the combined scoring model value is stored into thepreviously-referenced data structure along with the term or phrase.Further, in some embodiments, the combined scoring model value iscomputed separately and is not stored into the data structure. In someembodiments, the combined scoring model value may be based upon someother suitable expression used to determine the frequency of a phraseand the terms contained therein.

In some embodiments, the idf value may be more significant where thephrases used to describe a buyer or seller more closely approximates,for example, the universe of all words or phrases to describe sellers ingeneral. If “Happy_Shopper” is described as “An unparalleled Seller”three times in feedback related to them, and the phrase “An unparalleledSeller” is only used three times in the universe of all seller feedback,then the idf value will taken on greater significance. In some cases,the use of an idf value may ensure the uniqueness of a phrase relativeto the universe of phrases within which the phrase may be found. Thisidf value may also be used to determine the gradation based upon importof a phrase such that the phrase may be highlighted to a greater orlesser degree. For example, if the idf value is close to 1, then thephrase may appear bigger (e.g., larger font), be represented in a uniquecolor or have some other way of representing it as distinct from otherphrases that may appear on a feedback page such as feedback page 203.

FIG. 17 is a diagram 1700 illustrating the conversion of the previouslyillustrated tries (see FIG. 14) into the data structure (e.g., a hashtable) generated by the execution of the operation 1603. Shown are thepreviously-illustrated first sub-trie 1429, second sub-trie 1430, andthird sub-trie 1431. These tries, and the phrases contained therein, maybe parsed such that the phrases generated through the traversing of apath from a root node to the leaf node may be extracted. This extractionmay be on the basis of, for example, certain types of phrase extractionrules as outlined in FIG. 15. Once extraction occurs and a frequencycount is generated for, in some embodiments, the aggregation of thefrequency values for each word contained in a phrase, these phrases andassociated frequency counts are then placed into at least one positionin, for example, a hash table 1702. Shown is a first position 1703 thatcontains the phrase “Best Purchase Ever” and the corresponding countvalues associated with this phrase, which here is 2154. Further, asecond position 1704 in the hash table 1702 contains the phrase “ServiceWas About Average”, which has a frequency count of 123355. A thirdposition 1705 is illustrated that contains the phrase “Was AboutAverage” with a frequency count of 99565. Also illustrated is a fourthposition 1706 that contains the phrase “Not A Good Experience” with afrequency count of 5546, and a fifth position 1707 containing the phrase“A Good Experience” with a frequency count of 45363. In someembodiments, a combined scoring model value may be associated with thephrase and may be placed into a position within the hash table, eitherin lieu of or together with the frequency count. As previously stated,hash table 1702 may, in some embodiments, use bucket hashing, hashclustering or some other suitable embodiment or implementation of a hashtable.

FIG. 18 is a flowchart illustrating an example method used to executeoperation 813. Illustrated is an operation 1801 that extracts a scoringmodel from a data structure and discards certain phrases and modelsbelow a certain threshold value. As previously discussed, this scoringmodel may be, for example, the aggregation of scores contained in a hashtable, such as hash table 1702, wherein an entire hash table may bediscarded based upon the aggregation of the low scores associated withthe various entries into the hash table (e.g., frequency countsassociated with certain phrases). In other cases, certain specificentries in the hash table may be discarded based upon their lowfrequency count. The certain threshold value may be predetermined by,for example, a system administrator or other suitable individual. Uponthe successful execution of operation 1801, an operation 1803 may beexecuted that maps the remaining phrases and corresponding models to anemoticon database 1802 that may contain certain graphic illustrations(e.g., emoticons) representing the model. In some cases, the variousfrequency counts for a particular phrase may be mapped to certain valuescontained in emoticon database 1802. If a phrase has a frequency countof 1,000, and an emoticon has a threshold value of 1,000, then theemoticon may be used. Next, an operation 1804 is executed that retrievesan emoticon as a mapping use in generating a feedback page. Put anotherway, once the mapping occurs through the execution of operation 1803,then certain emoticons may be retrieved and used in the generation of,for example, a feedback page such as feedback page 203.

FIG. 19 is a flowchart illustrating an example method used to executeoperation 814. Shown is an operation 1901 that receives phrases andgraphic emoticons. Once the phrases and emoticons have been received, anoperation 1902 is executed that sorts these phrases and emoticons into afeedback page such as, for example, feedback page 203. An operation 1903is then executed that retrieves threshold instructions relating to, forexample, font size, highlighting, bolding, or other types of thresholdinstructions from, for example, a threshold instruction database 1904.Next, an operation 1905 is executed that modifies the font size and/orhighlighting for phrases on the feedback page, where certain thresholdsare met or exceeded. The threshold instructions for the thresholdinstruction database 1904 may be generated by, for example, a systemadministrator or other suitable person.

In some embodiments, the modification of the font size or other types ofhighlighting relating to a particular phrase may occur where thecombined scoring model results in some the generation of some value(e.g., the combined scoring model value). In cases where this valuefalls within one particular area of the gradation based upon import tohighlight, then the font size, color, or some other way ofdistinguishing the phrase form other phrases contained in the feedbackwill be applied. In some cases, the larger the combined scoring modelvalue, the more uniquely highlighted the phrase will be. For example,under the threshold instructions, phrases with a combined scoring modelvalue of >100 are entitled to receive 16 point font, while those <=100are entitled to only 12 point font. And again, pursuant to the thresholdinstructions, phrases with a value of >200 may be entitled to beinghighlighted in the color red, but those with a value of <200 areentitled to no special coloring other than black.

Example Storage

Some embodiments may include the various databases (e.g., 132, 1104, or1503) being relational databases, or in some cases On-Line AnalyticalProcessing (OLAP) based databases. In the case of relational databases,various tables of data are created, and data is inserted into and/orselected from these tables using SQL or some other database-querylanguage known in the art. In the case of OLAP databases, one or moremulti-dimensional cubes or hypercubes containing multidimensional data,which data is selected from or inserted into using MDX, may beimplemented. In the case of a database using tables and SQL, a databaseapplication such as, for example, MYSQL™, SQLSERVER™, Oracle 8I™, 10G™,or some other suitable database application may be used to manage thedata. In the case of a database using cubes and MDX, a database usingMultidimensional On Line Analytic Processing (MOLAP), Relational On LineAnalytic Processing (ROLAP), Hybrid Online Analytic Processing (HOLAP),or some other suitable database application may be used to manage thedata. These tables or cubes made up of tables, in the case of, forexample, ROLAP, are organized into a RDS or Object Relational DataSchema (ORDS), as is known in the art. These schemas may be normalizedusing certain normalization algorithms so as to avoid abnormalities suchas non-additive joins and other problems. Additionally, thesenormalization algorithms may include Boyce-Codd Normal Form or someother normalization or optimization algorithm known in the art.

FIG. 20 is an example RDS 2000 illustrating some of the example databasetables that may be used in one or more embodiments of the system andmethod illustrated herein. Shown is a user ID table 2001 that maycontain user ID data. This user ID data may be in the form of, forexample, a user handle, such as the previously-shown user handle“Happy_Shopper”, or may be some other type of identifier such as anumeric reference. In cases where a handle is used, the user ID may be,for example, a string data type or character data type. In cases where anumerical value is used, the user ID may be, for example, an integerdata type. Also shown is a feedback data table 2002 containing feedbackdata. The data contained in feedback data table 2002 may be, forexample, data illustrating or otherwise showing negative, neutral, orpositive feedback. This feedback data may be in the form of, forexample, a string data type, a Character Large Object (CLOB) data type,or even just a character data type. Also illustrated is a table 2003 inthe form of data category. In some embodiments, table 2003 may includevarious categories of data that may be used to characterize or otherwisecategorize particular types of feedback beyond just whether or not theparticular type of feedback is positive, negative, or neutral. Forexample, these categories may include comment text, a common usercategory, a rule category, a feedback score category, a category ofitem, a cost of item, or some other suitable category. A data categorymay be, for example, a string data type, a character data type, or someother suitable data type.

Also shown is an extraction rules for phrases table 2005 containingvarious extraction rules for certain phrases. In some embodiments, whilethe tables 2001 through 2003 may reside on, for example, the feedbackdata store 132, the extraction rules for phrases table 2005 may resideas a part of, for example, the phrase extraction rules database 1503.Extraction rules may include rules regarding the number of words thatmay make up a phrase that may be able to be extracted, the frequencywith which these words may appear across a plurality of phrases, suchthat these words may be used in a phrase that is extracted, or someother suitable set of rules. Also shown is a synonyms table 2007 thatmay reside as a part of the emotions database 1802. These rules may bein the form of strings or character data types wherein, for example, ifa negative phrase is determined to by synonymous with a string in theemotions database 1802, then that phrase may be associated with anemoticon. Also shown is an illustrations table 2008 that may reside on,for example, the emoticon database 1802. Illustrations table 2008 maycontain graphic illustrations in the form of, for example, emoticons.The illustrations table 2008 may contain, for example, a Binary LargeObject (BLOB) that contains a binary representation of the actualgraphic illustration (e.g., the emoticon).

Further, a stop word table 2006 is shown that may reside as a part of,for example, the noise word database 1104. Stop word table 206 may beoptional, but when used or otherwise implemented, it may contain aplurality of stop words such that if one of these stop words isencountered in a phrase, that word is then removed from the phrase or,more generally, from a particular piece of feedback data (e.g., 126, 127and/or 128). These stop words may be generated by, for example, a systemadministrator, and may be in the form of, for example, a string datatype. Further illustrated is a unique key table 2004 that provides aunique key value for one or more of the tables illustrated herein (e.g.,2001-2003, 2005-2008). These unique key values may be in the form of,for example, an integer or some other uniquely identifying numeric valueor plurality of unique identifying numeric values.

A Three-Tier Architecture

In some embodiments, a method is described as implemented in adistributed or non-distributed software application designed under athree-tier architecture paradigm, whereby the various components ofcomputer code that implement this method may be categorized as belongingto one or more of these three tiers. Some embodiments may include afirst tier as an interface (e.g., an interface tier) that is relativelyfree of application processing. Further, a second tier may be a logictier that performs application processing in the form oflogical/mathematical manipulations of data inputted through theinterface level, and communicates the results of theselogical/mathematical manipulations to the interface tier and/or to abackend, or storage tier. These logical/mathematical manipulations mayrelate to certain business rules, or processes that govern the softwareapplication as a whole. A third, storage tier, may be a persistent ornon-persistent storage medium. In some cases, one or more of these tiersmay be collapsed into another, resulting in a two-tier or even aone-tier architecture. For example, the interface and logic tiers may beconsolidated, or the logic and storage tiers may be consolidated, as inthe case of a software application with an embedded database. Thisthree-tier architecture may be implemented using one technology, or aswill be discussed below, a variety of technologies. This three-tierarchitecture, and the technologies through which it is implemented, maybe executed on two or more computer systems organized in aserver-client, peer to peer, or some other suitable configuration.Further, these three tiers may be distributed between more than onecomputer system as various software components.

Component Design

Some example embodiments may include the above described tiers, andprocesses or operations that make them up, as being written as one ormore software components. Common to many of these components is theability to generate, use, and manipulate data. These components, and thefunctionality associated with each, may be used by client, server, orpeer computer systems. These various components may be implemented by acomputer system on an as-needed basis. These components may be writtenin an object-oriented computer language such that a component oriented,or object-oriented programming technique can be implemented using aVisual Component Library (VCL), Component Library for Cross Platform(CLX), Java Beans (JB), Enterprise Java Beans (EJB), Component ObjectModel (COM), Distributed Component Object Model (DCOM), or othersuitable technique. These components may be linked to other componentsvia various Application Programming interfaces (APIs), and then compiledinto one complete server, client, and/or peer software application.Further, these APIs may be able to communicate through variousdistributed programming protocols as distributed computing components.

Distributed Computing Components and Protocols

Some example embodiments may include remote procedure calls being usedto implement one or more of the above described components across adistributed programming environment as distributed computing components.For example, an interface component (e.g., an interface tier) may resideon a first computer system that is located remotely from a secondcomputer system containing a logic component (e.g., a logic tier). Thesefirst and second computer systems may be configured in a server-client,peer-to-peer, or some other suitable configuration. These variouscomponents may be written using the above-described object-orientedprogramming techniques, and can be written in the same programminglanguage or in different programming languages. Various protocols may beimplemented to enable these various components to communicate regardlessof the programming language(s) used to write them. For example, acomponent written in C++ may be able to communicate with anothercomponent written in the Java programming language through use of adistributed computing protocol such as a Common Object Request BrokerArchitecture (CORBA), a Simple Object Access Protocol (SOAP), or someother suitable protocol. Some embodiments may include the use of one ormore of these protocols with the various protocols outlined in the OpenSystems Interconnection (OSI) model, or the Transmission ControlProtocol/Internet Protocol (TCP/IP) protocol stack model for definingthe protocols used by a network to transmit data.

A System of Transmission Between a Server and Client

Some embodiments may utilize the Open Systems Interconnection (OSI)basic reference model or Transmission Control Protocol/Internet Protocol(TCP/IP) protocol stack model for defining the protocols used by anetwork to transmit data. In applying these models, a system of datatransmission between a server and client, or between peer computersystems is described as a series of roughly five layers comprising: anapplication layer, a transport layer, a network layer, a data linklayer, and a physical layer. In the case of software having a three tierarchitecture, the various tiers (e.g., the interface, logic, and storagetiers) reside on the application layer of the TCP/IP protocol stack. Inan example implementation using the TCP/IP protocol stack model, datafrom an application residing at the application layer is loaded into thedata load field of a TCP segment residing at the transport layer. TheTCP segment also contains port information for a recipient softwareapplication residing remotely. The TCP segment is loaded into the dataload field of an IP datagram residing at the network layer. Next, the IPdatagram is loaded into a frame residing at the data link layer. Thisframe is then encoded at the physical layer, and the data is transmittedover a network such as an internet, Local Area Network (LAN), Wide AreaNetwork (WAN), or some other suitable network. In some cases, the word‘internet’ refers to a network of networks. These networks may use avariety of protocols for the exchange of data, including theaforementioned TCP/IP as well as ATM, SNA, SDI, or some other suitableprotocol. These networks may be organized within a variety of topologies(e.g., a star topology) or structures.

A Computer System

FIG. 21 shows a diagrammatic representation of a machine in the exampleform of a computer system 2100 within which a set of instructions forcausing the machine to perform any one or more of the methodologiesdiscussed herein may be executed. In alternative embodiments, themachine operates as a standalone device or may be connected (e.g.,networked) to other machines. In a networked deployment, the machine mayoperate in the capacity of a server or a client machine in server-clientnetwork environment, or as a peer machine in a peer-to-peer (ordistributed) network environment. The machine may be a Personal Computer(PC), a tablet PC, a Set-Top Box (STB), a Personal Digital Assistant(PDA), a cellular telephone, a web appliance, a network router, switchor bridge, or any machine capable of executing a set of instructions(sequential or otherwise) that specify actions to be taken by thatmachine. Further, while only a single machine is illustrated, the term“machine” shall also be taken to include any collection of machines thatindividually or jointly execute a set (or multiple sets) of instructionsto perform any one or more of the methodologies discussed herein.Example embodiments can also be practiced in distributed systemenvironments where local and remote computer systems that are linked(e.g., either by hardwired, wireless, or a combination of hardwired andwireless connections) through a network, both perform tasks. In adistributed system environment, program modules may be located in bothlocal and remote memory-storage devices (see below).

The example computer system 2100 includes a processor 2102 (e.g., aCentral Processing Unit (CPU), a Graphics Processing Unit (GPU) orboth), a main memory 2101 and a static memory 2106, which communicatewith each other via a bus 2108. The computer system 2100 may furtherinclude a video display unit 2110 (e.g., a Liquid Crystal Display (LCD)or a Cathode Ray Tube (CRT)). The computer system 2100 also includes analphanumeric input device 2117 (e.g., a keyboard), a User Interface (UI)cursor controller 2111 (e.g., a mouse), a disk drive unit 2116, a signalgeneration device 2153 (e.g., a speaker) and a network interface device(e.g., a transmitter) 2120.

The disk drive unit 2116 includes a machine-readable medium 2122 onwhich is stored one or more sets of instructions 2121 and datastructures (e.g., software) embodying or utilized by any one or more ofthe methodologies or functions described herein. The software may alsoreside, completely or at least partially, within the main memory 2101and/or within the processor 2102 during execution thereof by thecomputer system 2100, the main memory 2101 and the processor 2102 alsoconstituting machine-readable media.

The instructions 2121 may further be transmitted or received over anetwork 2126 via the network interface device 2120 utilizing any one ofa number of well-known transfer protocols (e.g., HTTP, SIP).

In some embodiments, a removable physical storage medium is shown to bea single medium, and the term “machine-readable medium” should be takento include a single medium or multiple media (e.g., a centralized ordistributed database, and/or associated caches and servers) that storethe one or more sets of instructions. The term “machine-readable medium”shall also be taken to include any medium that is capable of storing,encoding or carrying a set of instructions for execution by the machineand that cause the machine to perform any of the one or more of themethodologies described herein. The term “machine-readable medium” shallaccordingly be taken to include, but not be limited to, solid-statememories, optical and magnetic media, and carrier wave signals.

Marketplace Applications

In some embodiments, a system and method is illustrated that facilitatesthe generation of feedback that is useful and non-cumulative by removingredundant information or only providing useful feedback information.Feedback regarding a particular seller of a good or service may only beuseful insofar as it instructs a potential purchaser regarding specificinformation about a seller and/or the good or services that are beingsold. In cases where this feedback is merely cumulative, the feedback isnot informative. Feedback that is informative will attract morepotential purchasers, since these purchasers may be more able toresearch the sellers and the goods or services being sold on, forexample, a website.

The Abstract of the Disclosure is provided to comply with 37 C.F.R. §1.72(b), requiring an abstract that will allow the reader to quicklyascertain the nature of the technical disclosure. It is submitted withthe understanding that it will not be used to interpret or limit thescope or meaning of the claims. In addition, in the foregoing DetailedDescription, it can be seen that various features are grouped togetherin a single embodiment for the purpose of streamlining the disclosure.This method of disclosure is not to be interpreted as reflecting anintention that the claimed embodiments require more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus the following claims are herebyincorporated into the Detailed Description, with each claim standing onits own as a separate embodiment.

What is claimed is:
 1. A method comprising: receiving a feedback requestvia a user interface for a feedback page about a particular useridentifier; responsive to the feedback request, retrieving a pluralityof feedback entries submitted using a plurality of other useridentifiers and about the particular user identifier; determining afirst frequency for a first phrase extracted from the plurality offeedback entries; determining a second frequency for a second phraseextracted from the plurality of feedback entries; identifying, based onthe first frequency and the second frequency, a graphical illustrationthat is associated with both the first phrase and the second phrase; andgenerating the feedback page about the particular user identifier fordisplay via the user interface by inserting the graphical illustrationinto the feedback page.
 2. The method of claim 1, further comprisingdetermining that the first phrase is synonymous with one or more phrasesassociated with the graphical illustration.
 3. The method of claim 2,wherein determining that the first phrase is synonymous with the one ormore phrases includes comparing the first phrase to phrases associatedwith a plurality of graphical illustrations.
 4. The method of claim 1,further comprising determining that the first phrase and the secondphrase are synonymous with one or more phrases associated with thegraphical illustration.
 5. The method of claim 4, wherein determiningthat the first phrase and the second phrase are synonymous with the oneor more phrases includes comparing the first phrase and the secondphrase to phrases associated with a plurality of graphicalillustrations.
 6. The method of claim 1, wherein the graphicalillustration is associated with a threshold frequency such that thegraphical illustration is inserted into the feedback page when afrequency of phrases associated with the graphical illustration in theplurality of feedback entries satisfies the threshold frequency.
 7. Themethod of claim 6, further comprising determining that the firstfrequency for the first phrase and the second frequency for the secondphrase satisfies the threshold frequency associated with the graphicalillustration.
 8. The method of claim 1, wherein: a different graphicalillustration is associated with a respective threshold frequency; afrequency of phrases associated with the different graphicalillustration in the plurality of feedback entries does not satisfy therespective threshold frequency; and the phrases associated with thedifferent graphical illustration are discarded from use in generatingthe feedback page such that the different graphical illustration is notinserted into the feedback page.
 9. A system comprising: at least oneprocessor; and memory having stored thereon instructions that areexecutable by the at least one processor to perform operationsincluding: receiving a feedback request via a user interface for afeedback page about a particular user identifier; responsive to thefeedback request, retrieving a plurality of feedback entries submittedusing a plurality of other user identifiers and about the particularuser identifier; determining a first frequency for a first phraseextracted from the plurality of feedback entries; determining a secondfrequency for a second phrase extracted from the plurality of feedbackentries; identifying, based on the first frequency and the secondfrequency, a graphical illustration that is associated with both thefirst phrase and the second phrase; and generating the feedback pageabout the particular user identifier for display via the user interfaceby inserting the graphical illustration into the feedback page.
 10. Thesystem of claim 9, wherein the operations further include determiningthat the first phrase and the second phrase are synonymous with one ormore phrases associated with the graphical illustration.
 11. The systemof claim 10, wherein determining that the first phrase and the secondphrase are synonymous with the one or more phrases includes comparingthe first phrase and the second phrase to phrases associated with aplurality of graphical illustrations.
 12. The system of claim 9, whereinthe graphical illustration is associated with a threshold frequency suchthat the graphical illustration is inserted into the feedback page whena frequency of phrases associated with the graphical illustration in theplurality of feedback entries satisfies the threshold frequency.
 13. Thesystem of claim 12, wherein the operations further include determiningthat the first frequency for the first phrase and the second frequencyfor the second phrase satisfies the threshold frequency associated withthe graphical illustration.
 14. The system of claim 9, wherein: adifferent graphical illustration is associated with a respectivethreshold frequency; a frequency of phrases associated with thedifferent graphical illustration in the plurality of feedback entriesdoes not satisfy the threshold frequency; and the phrases associatedwith the different graphical illustration are discarded from use ingenerating the feedback page such that the different graphicalillustration is not inserted into the feedback page.
 15. Anon-transitory computer-readable storage medium comprising instructions,which when executed by one or more processors of a computing device,cause the computing device to perform operations including: receiving afeedback request via a user interface for a feedback page about aparticular user identifier; responsive to the feedback request,retrieving a plurality of feedback entries submitted using a pluralityof other user identifiers and about the particular user identifier;determining a first frequency for a first phrase extracted from theplurality of feedback entries; determining a second frequency for asecond phrase extracted from the plurality of feedback entries;identifying, based on the first frequency and the second frequency, agraphical illustration that is associated with both the first phrase andthe second phrase; and generating the feedback page about the particularuser identifier for display via the user interface by inserting thegraphical illustration into the feedback page.
 16. The computer-readablestorage medium of claim 15, wherein the operations further includedetermining that the first phrase and the second phrase are synonymouswith one or more phrases associated with the graphical illustration. 17.The computer-readable storage medium of claim 16, wherein determiningthat the first phrase and the second phrase are synonymous with the oneor more phrases includes comparing the first phrase and the secondphrase to phrases associated with a plurality of graphicalillustrations.
 18. The computer-readable storage medium of claim 15,wherein the graphical illustration is associated with a thresholdfrequency such that the graphical illustration is inserted into thefeedback page when a frequency of phrases associated with the graphicalillustration in the plurality of feedback entries satisfies thethreshold frequency.
 19. The computer-readable storage medium of claim18, wherein the operations further include determining that the firstfrequency for the first phrase and the second frequency for the secondphrase satisfies the threshold frequency associated with the graphicalillustration.
 20. The computer-readable storage medium of claim 15,wherein: a different graphical illustration is associated with arespective threshold frequency; a frequency of phrases associated withthe different graphical illustration in the plurality of feedbackentries does not satisfy the threshold frequency; and the phrasesassociated with the different graphical illustration are discarded fromuse in generating the feedback page such that the different graphicalillustration is not inserted into the feedback page.